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Abstract 

Building text planning resources by hand is time- 
consuming and difficult. Certainly, a number of 
planning architectures and their accompanying 
plan libraries have been implemented, but while 
the architectures themselves may be reused in a 
new domain, the library of plans typically cannot. 
One way to address this problem is to use ma- 
chine learning techniques to automate the deriva- 
tion of planning resources for new domains. In 
this paper, we apply this technique to build micro- 
planning rules for preventative expressions in in- 
structional text. 

1 Introduction 

Building text planning resources by hand is time- 
consuming and difficult. Certainly, much work 
has been done in this regard; there are a num- 
ber of freely available text pla nning architectures 
(e.g., ( Moore and Paris, 1993| )). It is frequently 
the case, however, that while the architecture it- 
self can be reused in a new domain, the library 
of text plans developed for it cannot. In particu- 
lar, micro-planning rules, those rules that spec- 
ify the low-level grammatical details of expres- 
sion, are highly sensitive to variations between 
sub-languages, and are therefore difficult to reuse. 

When faced with a new domain in which to gen- 
erate text, the typical scenario is to perform a 
corpus analysis on a representative collection of 
the text produced by human authors in that do- 
main and to induce a set of micro-planning rules 
guiding the generation process in accordance with 
the results. Some fairly simple rules usually jump 
out of the analysis quickly, mostly based on the 
analyst's intuitions. For example, in written in- 
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structions, user actions are typically expressed as 
imperatives. Such observations, however, tend to 
be gross characterisations. More accurate micro- 
planning requires painstaking analysis. In this pa- 
per, for example, the micro-planner must distin- 
guish between phrasing such as "Don't do action- 
X" and "Take care not to do action-X" . Without 
analysis, it is far from clear how this decision can 
best be made. 

Some form of automation would clearly be de- 
sirable. Unfortunately, corpus analysis techniques 
are not yet capable of automating the initial 
phases of the corpus study (nor will they be for 
the foreseeable future). There are, however, tech- 
niques for rule induction which are useful for the 
later stages of corpus analysis and for implemen- 
tation. 

In this paper, we focus on the use of such rule 
induction techniques in the context of the micro- 
planning of preventative expressions in instruc- 
tional text. We define what we mean by a pre- 
ventative expression, and go on to describe a cor- 
pus analysis in which we derive three features that 
predict the grammatical form of such expressions. 
We then use the C4.5 learning algorithm to con- 
struct a micro-planning sub-network appropriate 
for these expressions. We conclude with an im- 
plemented example in which the technical author 
is allowed to set the relevant features, and the 
system generates the appropriate expressions in 
English and in French. 



2 Preventative Expressions 

Preventative expressions are used to warn the 
reader not to perform certain inappropriate or po- 
tentially dangerous actions. The reader may be 
told, for example, "Do not enter" or "Take care 
not to push too hard" . Both of these examples 
involve negation ( "do not" and "take care not" ) . 
Although this is not strictly necessary for preven- 
tative expressions (e.g., one might say "stay out" 
rather than "do not enter" ) , we will focus on the 
use of negative forms in this paper, using the fol- 



lowing categorisation:^] 



• negative imperatives proper (termed DONT 
imperatives) — These are characterised by 
the negative auxiUary do not or don't ^ as in: 

1. Your sheet vinyl floor may be vinyl as- 
bestos, which is no longer on the market. 
Don't sand it or tear it up because this 
will put dangerous asbestos fibers into 
the air. 

• NEVER imperatives — These are charac- 
terised by the use of the negative adverb 
never ^ as in: 

1. Whatever you do, never go to Vienna if 
you are on a diet. 

• other negative imperatives (termed neg-TC 
imperatives) — These include take care and 
he careful followed by a negative infinitival 
complement, as in the following examples: 

1. To book the strip, fold the bottom third 
or more of the strip over the middle of 
the panel, pasted sides together, taking 
care not to crease the wallpaper sharply 
at the fold. 

2. If your plans call for replacing the wood 
base molding with vinyl cove molding, he 
careful not to damage the walls as you 
remove the wood base. 

3 Corpus Analysis 

In terms of text generation, our interest is in find- 
ing mappings from features related to the func- 
tion of these expressions, to those related to their 
grammatical form. Functional features include 
the semantic features of the message being ex- 
pressed, the pragmatic features of the context of 
communication, and the features of the surround- 
ing text being generated. In this section we will 
briefly discuss the nature of our corpus, and the 
function and form features that we have coded. 
We will conclude with a discussion of the inter- 
coder reliability. A more detailed discussion of 



this portion of the work is given elsewhere ((Van- 
dcr Linden and Di Eugenio, 1996)). 



3.1 Corpus 

The corpus from which we take all our coded ex- 
amples has been collected opportunistically off the 
internet and from other sources. It is 4.5 MB in 
size and is made entirely of written English in- 
structional texts. As a collection, these texts are 
the result of a variety of authors working in a va- 
riety of contexts. 



We broke the corpus texts into expressions us- 
ing a simple sentence breaking algorithm and then 
collected the negative imperatives by probing for 
expressions that contain the grammatical forms 
we were interested in (i.e., expressions containing 
phrases such as don't, never, and take care). The 
grammatical forms we found, 1283 occurrences in 
all, constitute 2.7% of the expressions in the full 
corpus. The first line in Table |l|, marked "Raw 
Grcp" , indicates the quantity of each type. 

We then filtered the results. When the probe re- 
turned more than 100 examples for a grammatical 
form, we randomly selected around 100 of those 
returned, as shown in line 2 of Table ^ (labelled 
"Raw Sample"). We then removed those exam- 
ples that, although they contained the desired lex- 
ical string, did not constitute negative imperatives 
(e.g., "If you don't like the colors of the file, . . . , 
use Binder to change them."), as shown in line 3, 
labelled "Final Coding" . 

The final corpus sample is made up of 279 exam- 
ples, all of which have been coded for the features 
to be discussed in the next two sections. Table || 
also shows the relative sizes of the various types 
of instructions in the corpus as well as the number 
of examples from this sample that came from each 
type. 

3.2 Form 

Because of its syntactic nature, the form feature 
coding was very robust. The possible feature val- 
ues were: DONT — for the do not and don't 
forms discussed above; NEVER, for imperatives 
containing never] and neg-TC — for take care, 
make sure, he careful, and he sure expressions with 
negative arguments. The two authors agreed on 
their coding of this feature in all cases. 

3.3 Function Features 

We will now briefly discuss three of the func- 
tion features we have coded: intentionality, 
AWARENESS, and SAFETY. We illustrate them in 
turn using a to refer to the prevented action and 
using "agent" to refer to the reader and executer 
of the instructions. 

Intentionality: This feature encodes whether 
or not the writer believes that the agent will con- 
sciously adopt the intention of performing a: 

CON is used to code situations where the agent 
intends to perform a. In this case, the agent 
must be aware that a is one of his or her 
possible alternatives. 

UNC is used to code situations in which the agent 
doesn't realize that there is a choice involved 



^Horn ((1989)) gives a more complete categorisa- 
tion of negative forms. 



^Note that we us ed a n umber of examples from 
Di Eugenio's thesis ((L993b)) which were included as 



excerpts. In this table we include only an estimate of 
the full size of that portion of the corpus. 





DONT 


NEVER 


Neg-TC 




don't 


do not 




take care 


make sure 


be careful 


be sure 


Raw Grcp 


417 


385 


108 


21 


229 


52 


71 


Raw Sample 


100 


99 


108 


21 


104 


52 


71 


Final Coding 


78 


89 


40 


17 


3 


46 


6 




167 


40 


72 



Table 1: Distribution of negative imperatives 



Instruction type 


Corpus size 


# of preventatives 


Recipes 


1.7M 


83 


Do-it-yourself 


1.26M 


99 


Di Eugenio's thesis^ 


336K 


69 


Software instructions 


264K 





Administrative forms 


317K 


9 


Other 


565K 


19 


Totals 


4.5M 


279 



Table 2: Distribution of examples from sample 



(cf. (pi Eugenio, 1993a)). It is used in two 1988| )) as a measure of coder agreement. For nom- 



situations: when a is totally accidental, or 

Awareness: This feature captures whether or 
not the writer believes that the agent is aware 
that the consequences of a are bad: 

AW is used when the agent is aware that a is 
bad. For example, the agent may be told "Be 
careful not to burn the garlic" when he or she 
is perfectly well aware that burning things 
when cooking them is bad. 

UNAW is used when the agent is perceived to be 
unaware that a is bad. 

Safety: This feature captures whether or not 
the author believes that the agent's safety is put 
at risk by performing a: 

BADP is used when the agent's safety is put at 
risk by performing a. 

NOT is used when it is not unsafe to perform a, 
but may, rather, be simply inconvenient. 

3.4 Inter-coder reliability 

Each author independently coded each of the fea- 
tures for all the examples in the sample. The 
percentage agreement for each of the features is 
shown in the following table: 



feature 


percent agreement 


form 


100% 


intentionality 

awareness 

safety 


74.9% 
93.5% 
90.7% 



As advocated by Carletta (( 1996])), we have 
used the Kappa coefficient (( ^iegel and Castellan ' 



inal data, this statistic not only measures agree- 
ment, but also factors out chance agreement. 

If P{A) is the proportion of times the coders 
agree, and P{E) is the proportion of times that 
coders are expected to agree by chance, K is com- 
puted as follows: 



K = 



P{A) - P{E) 
1 - P{E) 



There are various ways of computing P{E) ac- 
cording to Siegel and Castellan (( 1988| )); most re- 
searchers agree on the following formula, which we 
also adopted: 

m 

P{E) ^ 

where m is the number of categories, and pj is the 
proportion of objects assigned to category j. 

The mere fact that K may have a value k greater 
than zero is not sufficient to draw any conclusion, 
however, as it must be established whether k is 
significantly different from zero. There are sug- 
gestions in the literature that allow us to draw 
general conclusions without these further com- 
putations. For example, Rietveld and van Hout 



((1993)) suggest the correlation between K values 
and inter-coder reliability shown in the following 
table: 



Kappa Value 


Reliability Level 


.00 ~ .20 


slight 


.21 - .40 


fair 


.41 - .60 


moderate 


.61 - .80 


substantial 


.81 - 1.00 


almost perfect 



For the form feature, the Kappa value is 1.0, indi- 
cating perfect agreement. The function features, 



which arc more subjective in nature, engender 
more disagreement among coders, as shown by the 
K values in the following table: 



feature 


K 


INTENTIONALITY 

AWARENESS 

SAFETY 


0.46 
0.76 
0.71 



According to this table, therefore, the awareness 
and safety features show "substantial" agree- 
ment and the intentionality feature shows 
"moderate" agreement. We have coded other 
functional features as well, but they have either 
not proven as reliable as these, or are not as use- 
ful in text planning. 



In addition, Siegel and Castellan ((1988)) point 
out that it is possible to check the significance of K 
when the number of objects is large; this involves 
computing the distribution of K itself. Under this 
approach, the three values above are significant at 
the .000005 level. 

4 Automated Learning 

The corpus analysis results in a set of examples 
coded with the values of the function and form 
features. This data can be used to find correla- 
tions between the two types of features, correla- 
tions, which, in text generation, are typically im- 
plemented as decision trees or rule sets mapping 



the examples reserved for the training set and the 
testing set respectively. The upper stream pro- 
cesses the training set and contains a type module 
which marks the main syntactic form (i.e., DONT, 
NEVER, or Neg-TC) as the variable to be pre- 
dicted and the awareness, safety, and inten- 
tionality features as the inputs. Its output is 
passed to the C4.5 node, labelled mform, which 
produces the decision tree. We then use two copies 
of the resulting decision tree, represented by the 
diamond shaped nodes marked with mform, to 
test the accuracy of the testing and the training 
sets. 

One run of the system, for example, gave the 
following decision tree: 

awareness = AW: NEG-TC 

awareness = UNAW: 

I intention = CON: DONT 

I intention = UNC: 

I I safety = BADP: NEVER 

I I safety = NOT: DONT 

This tree takes the three function features and 
predicts the DONT, NEVER, and Neg-TC forms. 
It confirms our intuitions that never imperatives 
are used when personal safety may be endan- 
gered (coded as safety= "BADP" ) , and that Neg- 
TC forms are used when the reader is expected to 
be aware of the danger that may arise (cf. (Van- 



from function features to forms. 



der Linden and Di Eugenio, 1996)). It accurately 



In this study, we used 179 coded examples as 
input to the learning algorithm. These are the 
examples on which the two authors agreed on their 
coding of all the features. The distribution of the 
grammatical forms in these examples is shown in 
the following table: 



form 


frequency 


DONT 

Neg-TC 

NEVER 


100 
57 
22 



The learning algorithm used these examples to de- 
rive a decision tree which we then integrated into 
an existing micro-planner. 

4.1 Data Mining 

We hav e used Quinlan's C4.5 learning algorithm 
(( |l993[ )) in this study; this algorithm can induce 
either decision trees or rules. To provide a more 
convenient learning environment, we have used 
Clementine (( |1995D ), a tool which allows rapid 
reconfiguration of various data manipulation fa- 
cilities, including C4.5. Figure |l| shows the basic 
control stream we used for learning and testing de- 
cision trees. Data is input from the split-output 
file node on the left of the figure and is passed 
through filtering modules until it reaches the out- 
put modules on the right. The two select mod- 
ules (pointed to by the main input node) select 



predicts the grammatical form of 74.5% of the 161 
training examples, and 83.3% of the 18 testing ex- 
amples. 

Because there are relatively few training exam- 
ples in our coded corpus, we have also performed a 
10- way cross-validation test.^ None of the derived 
trees in this test were remarkably different from 
the one just shown, although they did order the 
intentionality and awareness features differ- 
ently. The average accuracy of the learned deci- 
sion trees on the testing sets was 75.4%. 

Note that although this level of accuracy is bet- 
ter than 55.9%, the score achieved by simply se- 
lecting DONT in all cases, there is still more work 
to be done. The current features must be refined, 
and more features may be need to be added. We 
are currently experimenting with a number of pos- 
sibilities. Note also that we have not distinguished 
between the various sub-forms of DONT and Neg- 
TC shown in Table |l]; this will require yet more 
features. 

Clementine can also "balance" the input to 
C4.5 by duplicating training examples with under- 
represented feature values. We used this to in- 
crease the number of NEVER and Neg-TC exam- 

•^A cross-validation test is a test where C4.5 breaks 
the data into different combinations of training and 
testing sets, builds and te sts decisio n trees for each, 
and averages the results ((Cle, 1995)). 




select nf orn analysis 



Figure 1: The Clementine learning environment 



pies to match the number of DONT examples. Ul- 
timately, this reduced the accuracy of the learned 
trees to 68.0% in a cross-validation test. The re- 
sulting decision trees tended not to include all 
three features. 

4.2 Integration 

Because it is common for us to rebuild decision 
trees frequently during analysis, we implemented 
a routine which automatically converts the deci- 
sion tree into the appropriate KPML-style sys- 
tem networks with their associated choosers, in- 



quiries, and inquiry implementations ((Bateman 
1995| )). This makes the network compatible with 



the DRAFTER micro-plauuer, a descendent of IM- 
AGENE ( dVander Linden and Martin, 1995| )). The 
conversion routine takes the following inputs: 

• the applicable language(s) — C4.5 produces 
its decision trees based on examples from a 
particular language, and KPML is capable 
of being conditionalised for particular lan- 
guages. Thus, we may perform separate cor- 
pus analyses of a particular phenomenon for 
various languages, and learn separate micro- 
planning trees; 

• the input feature(s) — The sub-network be- 
ing built must fit into the overall categorisa- 
tions of the full micro-planner, and thus we 
must specify the text functions that would 
trigger entry to the new sub-network; 

• the decision tree itself; 

• a feature-value function — To traverse the 
new sub-network, the KPML inquiries require 
a function that can determine the value of the 
features for each pass through the network; 

• grammatical form specifications — The sub- 
network must eventually build sentence plan 
language (SPL) commands for input to 



KPML, and thus must be told the appropri- 
ate SPL terms to use to specify the required 
grammatical forms; 

• an output file name. 

For our example, the system sub-network shown 
in Figure |^ is produced based on the decision tree 
shown above.^ It is important to note here that al- 
though the micro-planner is implemented as a sys- 
temic resource, the machine learning algorithm is 
no respecter of systemic linguistic theory. It sim- 
ply builds decision trees. This gives rise to three 
distinctly non-systemic features of these learned 
networks: 

1. The realisation statements are included only 
at the leaf nodes of the network. We have 
built no intelligent facility for decomposing 
the realisation statements and filtering com- 
mon realisations up the tree. 

2. The learning algorithm will freely reuse sys- 
tems (i.e., features) as various points in the 
tree. This did not happen in Figure ^, but oc- 
casionally one of the features is independently 
used in different sub-trees of the network. We 
are forced, therefore, to index the system and 
feature names with integers to disambiguate. 

3. There is no meta- functional distinction in the 
network, but rather, all the features, regard- 
less of their semantic type, are included in the 
same tree. 

The sub-network derived in this section was 
spliced into the existing micro-planning network 
for the full generation system. As mentioned 
above, this integration was done by manually 



''Only the systems are shown in the KPML dump 
given in Figure |2| The realisation statements, 
choosers, inquiries, and inquiry implementations are 
not shown. 



^AWARE-1 
'^UNAWARE-2'' 



CONSCIOUSNESS-SYSTEM-3 



CONSCIOUS-4 
UNCONSCIOUS-5'' 



SAFETY-SYSTEM-6 



.NOT-BADP-7 
"^BADP-S I 



Figure 2: The micro-planner system network derived from the decision tree 



specifying the desired input conditions for the sub- 
network when the micro-planning rules are built. 
For the preventative expression sub-network, this 
turned out to be a relatively simple matter. 
drafter's model of procedural relations includes 
a warning relation which may be attached by the 
author where appropriate. The micro-planner, 
therefore, is able to identify those portions of the 
procedure which are to be expressed as warnings, 
and to enter the derived sub-network appropri- 
ately. This same process could be done with any 
of the other procedural relations (e.g., purpose, 
precondition). This assumes, however, the exis- 
tence of a core set of micro-plans which perform 
the procedural categorisation properly; these were 
built by hand. We have only just begun to ex- 
periment with the possibility of building the en- 
tire network automatically from a more exhaustive 
corpus analysis. 

5 A DRAFTER Example 

Given the corpus analysis and the learned sys- 
tem networks discussed above, we will present an 
example of how preventative expressions can be 
delivered in drafter, an implemented text gen- 
eration application, drafter is a instructional 
text authoring tool that allows technical authors 
to specify a procedural structure, and then uses 
that structure as input to a multilingual text ge n- 
eration facility ( ( Paris and Vandcr Linden, 1996 ) ) . 
The instructions are generated in English and in 
French. 

To date, our domain of application has been 
manuals for software user interfaces, but because 
this domain does not commonly contain preventa- 
tive expressions (see Table |^), we have extended 
drafter's domain model to include coverage for 
do-it-yourself applications. Although this switch 
has entailed some additions to the domain model, 
drafter's input and generation facilities remain 
as they were. 

5.1 Input Specification 

In drafter, technical authors specify the content 
of instructions in a language independent man- 
ner using the drafter specification tool. This 
tool allows the authors to specify both the propo- 
sitional representations of the actions to be in- 
cluded, and the procedural relationships between 
those propositions. Figure ^ shows the drafter 
interface after this has been done. We will use 
the procedure shown there as an example in this 



section, det ails on how to build it can be found 
elsewhere ( ([Paris and Vandcr Linden, 1996| )). 

The INTERFACE and ACTIONS panes on the 
left of figure § list all the objects and actions de- 
fined so far. These are all shown in terms of a 
pseudo-text which gives an indication, albeit un- 
grammatical, of the nature of the action. For 
example, the main goal, "repair device" , repre- 
sents the action of the reader repairing an arbi- 
trary device. This node may be expressed in any 
number of different grammatical forms depending 
upon context. 

The WORKSPACE pane shows the procedure, 
represented in an outline format. The main user 
goal of repairing the device is represented by the 
largest, enclosing box. Within this box, there is 
a single method, called "Repair Method" which 
details how the repair should be done. There are 
three sub-actions: consulting the manual, unplug- 
ging the device, and removing the cover. There is 
also a warning slot filled with the action "[reader] 
damage service cover". This indicates that the 
reader should avoid damaging the service cover .| 

Neither the propositional nor the procedural in- 
formation discussed so far specify the three fea- 
tures needed by the decision network derived in 
the previous section (i.e., intentionality, aware- 
ness, and safety). At this point, we see no straight- 
forward way in which they could be determined 
a utoma tically (see Ansari's discussion of this issue 
(( 1995|) )). We, therefore, rely on the author to set 
them manually, drafter allows authors to set 
generation parameters on individual actions using 
a dialog box mechanism. Figure ^ shows a case 
in which the author has marked the following four 
features for the warning action "damage service 
cover" : 

• The action is to be prevented, rather than 
ensured; 

• Performing the action would result in incon- 
venience, but not in personal danger; 

• The user is likely to do the action acciden- 
tally, rather than consciously; 

• The user is likely to be aware that performing 
the action would create problems; 



^Actually, this could also be interpreted as an 
ensurative warning, meaning that the reader should 
make sure to damage the service cover (although this 
is clearly nonsensical in this case). We have not yet 
analysed such expressions and thus do not support 
them in drafter. 



INTERFACE 
Test nevicK Program 

Import VisualWorks| 



ACTIONS 
Repair Device 

Consult Repair Manm 

Unplug Device 

Remove Service Cove. 
Damage Service Cover 
Start Test Device Progn 
Quit Test Device Progra. 

Newl 



WORKSPACE 



repair device 



Methods 



Repair l^etiiod 

Warning damage service cover 

3ub-5tep5 i^. consult repair manual ^ unplug device remove service cover 



Finder 



Plan 



Repair Method 



FOCUS 



Repair f/letiyod 

Precondition 

Side-effect 

Cancellation 

Warning damage service cover 

Sub-steps ^ consult repair manual ^J- unplug device ^ remove 



Figure 3: drafter screen with the procedural structure for the example 



5.2 Text Generation 

Once the input procedure is specified, the author 
may initiate text generation from any node in the 
procedural hierarchy. When the technical author 
generates from the root goal node in Figure ^, for 
example, the following texts are produced: 
English: 

To repair the device 

1. Consult the repair manual. 

2. Unplug the device. 

3. Remove the service cover. 

Take care not to damage the service 
cover. 



Note that the French version employs eviter 
{avoid) rather than the less common prendre soin 
de ne pas {take care not). This is possible be- 
cause the French text is produced by a separate 
micro-planning sub-network. This sub-network 
was not based on a corpus study of French preven- 
tatives, but rather was implemented by taking the 
learned English decision tree, modifying it in ac- 
cordance with the intuitions of a French speaker, 
and automatically constructing French systems 
from that modified decision tree. Clearly, a cor- 
pus study French of preventatives is still needed, 
but this does show drafter's ability to make use 
of KPML's language conditionahsed resources. 



French: 

Reparation du dispositif 

1. Se reporter au manuel de reparation. 

2. Debrancher le dispositif. 

3. Enlever le couvercle de service. 
Eviter d'endommager le couvercle de ser- 
vice. 



Were we to replace the warning with other sorts 
of warnings, the expression would also change ac- 
cording to the learned micro-planning network. If 
authors, for example, wish to prevent the reader 
from performing the action of dismantling the 
frame of the device, and they decide that the 
reader is unaware of this danger, that the action is 
consciously performed and not unsafe, drafter 
produces the following text: 



mm 



What type of warning is ttnis? 

Wlnat are tine consequences of ignoring it? 

Is the user iil^ely to do it on purpose? 

Is the user iil^ely to be aware of this probiem? 



io,n features 




prevent the action ensure the action 



♦ inconuenience serious danger 



•0" on purpose by accident 



unaware ♦ aware 



ok) [Cancel 



Figure 4: The DRAFTER dialog box for setting the local parameters 



Do not dismantle the frame. 
Ne pas demonter I'armature. 

If authors wish to prevent the reader from dis- 
connecting the ground connection, and they de- 
cide that the reader is unaware of this danger, that 
the action would be unconsciously performed, and 

that the consequences are indeed life-threatening, 
DRAFTER produccs thc following text: 

Never disconnect the ground. 

Ne jamais deconnecter la borne de terre. 

6 Conclusion 

In this paper we have discussed the use of machine 
learning techniques for thc automatic construc- 
tion of micro-planning sub- networks. We demon- 
strated this for the case of preventative expres- 
sions in instructional text. 

We noted that because the automatic derivation 
of useful, well-defined features for corpus anal- 
ysis is beyond thc current state of thc art, thc 
painstaking process of corpus analysis must still 
be performed manually. As an example of how 
this can be done, we presented an analysis of En- 
glish preventative expressions. We intend to con- 
tinue this part of the work by addressing more 
preventative forms, addressing ensurativc forms, 
and by extending the analysis to other languages. 

Although the analysis cannot be fully auto- 
mated, we noted that the derivation of decision 
networks from coded corpus examples can. This 
greatly simplifies the tasks of building and testing 
text planning resources for new domains. We in- 
tend to continue this part of the work by applying 
the technique to larger portions of the planning 
resources. 
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